du dn

Background information: Recently, a lot of datanode node disk IO is relatively high, the main reason is due to the increase in job number, as well as the size of the increase.But any way to reduce disk IO consumption, we can all try. For example,

This section mainly introduces the data storage and management in the DN, we know that logically we store the data into the HDFs file system, but the specific data on each DN is how to store it, which involves several relatively large class

We can use two methods to obtain CPU and memory information: use the top tool provided by Linux, or directly read the directory/proc/{process ID}/stat in the file system. Here, I will introduce another method to obtain this information, which is

  Name Value Description DFS. Default. Chunk. View. Size 32768 The size of each file displayed on the HTTP access page of namenode usually does not need to be set. DFS. datanode. Du. Reserved 1073741824

Prepare for Work [Root@centos-01 ~]# cd/tmp/[root@centos-01 tmp]# ls 1.txt Systemd-private-40c0e692674844949b91361dc6ab4a40-chronyd.service-6km7k9 Systemd-private-40c0e692674844949b91361dc6ab4a40-vgauthd.service-3w1kyg Systemd-private-40c0e6926748

1. io. file. buffer. size is used to set the cache size for IO operations. The unit is byte. The default value is 4 kb. It is recommended to set it to 64 KB, that is, 655362. dfs. balance. the bandwidth of the bandwithPerSec cluster between the dn

If you want to know the source code of HDFS, you can read his javaeye from Brother Cai bin.  Sorry, I used the word "mysterious killer" because it really hurt me so much that I took a lot of energy to pick it up. Recently, when testing hadoop, The

When testing hadoop, The dfshealth. jsp Management page on the namenode shows that during the running of datanode, the last contact parameter often exceeds 3. LC (last contact) indicates how many seconds the datanode has not sent a heartbeat packet

Catalogue What is HDFs? Advantages and disadvantages of HDFs The framework of HDFs HDFs Read and write process HDFs command HDFs parameters 1. What is HDFsThe HDFS (Hadoop Distributed File System) is the core

I. INTRODUCTION to psql is a PostgreSQL command line interactive client tool, similar to the command line tool sqlplus: 1 in Oracle allows you to interactively type SQL or commands, then they are sent to the PostgreSQL server, and then the SQL or